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ABSTRACT 

Purpose: The environment of public health practice is rapidly changing, creating the need to adapt graduate 
education and accelerate educational innovation. Formative peer review is a strategy designed to promote 
critical reflection on teaching and to develop faculty as teachers. Through case study methods, we explore how 
peer review of teaching may catalyze reflective practice and contribute to the redesign of public health 
education. Methods: We conducted a detailed contextual analysis of a peer review of teaching program 
implemented from 2011-2014 in a global health department with approximately 200 students and 26 teaching 
faculty. The case study used multiple data sources including peer review feedback reports, a survey of 
participating instructors and reviewers, and administrative data. Results: Faculty had favorable attitudes toward 
peer review, especially the opportunity to learn by observing others teach. Peer review reports confirmed 
existing good teaching practices and suggested ways to improve student engagement and develop teachers in 
ways which are aligned with MPH program redesign goals. Both peer reviewers and instructors reported 
benefiting from the process. Conclusions: Formative peer review may help to spread innovative teaching 
practices among faculty by providing opportunities for reflection, increasing motivation and confidence to adopt 
pedagogical changes, and strengthening teaching community. Increased knowledge of other teachers’ courses 
and social ties created through peer review can be resources in promoting MPH curriculum integration and 
collaboration across disciplines. Recommendations: Schools of public health should consider peer review of 
teaching as a pathway to promote greater student engagement and practice-based learning. In planning for peer 
review of teaching programs, leaders need to find ways to address time-related concerns and align the peer 
review process with other efforts to promote program redesign goals and strengthen teaching community. 
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INTRODUCTION 

The curriculum for the Masters of Public 
Health (MPH) degree program is undergoing 
significant changes. In 2011, a task force of the 
Association of Schools and Programs of Public 
Health (ASPPH) studied the role of public health 
education for the 21 st century (Petersen & Weist, 
2014). The task force emphasized the need to 
adapt education to a rapidly changing 
environment in public health practice and to 
accelerate educational innovation. The skills 
which students of public health need are 
interdisciplinary, interprofessional, and systems 
oriented (Frenk et al., 2010; MPH Expert Panel 
of the Framing the Future Task Force, 2014; 
Petersen & Weist, 2014). They reflect work-force 
competencies from the core disciplines of 
biostatistics, epidemiology, health policy and 
management, environmental health sciences, 
and social and behavioral sciences in addition to 
cross-cutting skills in communication, change 
management, informatics, political and cultural 
sensitivity, and leadership (Bernstein, Jonson, & 
Smith, 2000; McKee & Tew, 2013). The new 
focus on applied learning and integration was 
expected to require major changes in teaching 
(Frenk, Hunter, & Lapp, 2015). 

Research supports the assumption that the 
quality of teaching affects student learning, and 
continuous development of faculty as teachers is 
critically important to successful education 
redesign (Bernstein et al., 2000; Clayson, 2009; 
McKee & Tew, 2013; MPH Expert Panel of the 
Framing the Future Task Force, 2014). To 
achieve the goals proposed by the ASPPH task 
force, course design and teaching must change 
in ways that promote student engagement and 
practice-based learning to build competencies in 
public health, with more time devoted to 
applying skills such as data analysis or critical 
reasoning, and practicing communications and 
leadership (Brame, 2013; Frenk et al., 2015; 
Fulton, 2013; Herreid & Schiller, 2013). Yet to 
date, most of the literature on redesigning the 
MPH has focused on curriculum choices (what 
to teach) rather than teaching methods and 
professional development of teaching faculty 
(how to teach). 

Peer review of teaching (PRT) is one method 
of promoting professional development of 
teachers, encouraging reflective practice, and 
motivating faculty to teach from a different 
perspective (Thomas, Chie, Abraham, Raj, & 
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Beh, 2014). PRT is defined as “informed 
colleague judgment about teaching for either 
fostering improvement or making personnel 
decisions” (Chism, 2007). Formative PRT 
programs are designed to develop faculty 
members’ self-awareness and provide teachers 
with information and an opportunity for critical 
reflection to improve their teaching practice, 
while evaluative PRT programs seek to compare 
teaching to standards to make judgments 
related to hiring, promotion, tenure, or salary 
review (Chism, 2007; Thomas et al., 2014). PRT 
can help teachers reflect on weaknesses and 
build on teaching strengths, select teaching 
methods that can best meet the goals for a 
course, develop more practice-based exercises 
and assessments, and integrate technology into 
the classroom (Carter, 2010; Kohut, Burnap, & 
Yon, 2010; Thomas et al., 2014; Yon, Burnap, & 
Kohut, 2010). Faculty peers may be better 
qualified than students to assess the 
appropriateness of course objectives, quality of 
course content, whether important points are 
being emphasized, and the aptness of examples 
used in class (Courneya, Pratt, & Collins, 2008; 
d'Eon, Overgaard, & Harding, 2000; Nelson, 
1998). The Harvard T.H. Chan School of Public 
Health includes peer review of teaching as a 
component of their new educational agenda, 
suggesting that it can be a vehicle to adapt the 
values of collaborative, team-based research to 
the teaching enterprise (Frenk et al., 2015). Yet, 
little is known about the application of peer 
review of teaching in the context of MPH 
education. Implementation of PRT programs in 
other disciplines has proven challenging 
because the goals and format must be adapted 
to institutional context, including the 
department’s mission, values, student 
demographics, resources, and faculty disciplines 
(Chism, 2007; Quinlan & Akerlind, 2000). Other 
obstacles or barriers to successful program 
operation include fear on the part of the 
instructor being reviewed, uncertainty about 
what to review, resistance to innovation that 
involves participants doing more work, and 
implementation factors such as how reviewers 
are chosen, power differentials, and the amount 
of guidance or flexibility built into the process 
(Costello, Pateman, Pusey, & Longshaw, 2001; 
Courneya et al., 2008; Frenk et al., 2010; Kell & 
Annetts, 2012; Thomas et al., 2014). We 
examine issues encountered in using PRT to 
help faculty in one department reflect on 
teaching practice, and explore what role PRT 
might play in accelerating educational innovation 
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in MPH programs across departments and 
schools of public health. 

PURPOSE 

The purpose of this study is to explore how a 
global health department at a major school of 
public health implemented PRT in conjunction 
with MPH curriculum redesign, in an effort to 
improve teaching which would reinforce 
competency-based learning. The study 
documented the views and perceptions of 
faculty about the PRT process, the types of 
feedback and observations gleaned through 
PRT, and the challenges in carrying out the 
program. The insight generated from this study 
may help envision the role of PRT in promoting 
innovation in MPH education, and inform the 
design and implementation of PRT across 
departments and in other public health 
institutions. 

Background on the Peer Review of Teaching 
Policy and Process 

The Department of Global Health (GH) 
admits 90-110 new MPH students per year. Full¬ 
time students generally can complete 
coursework in three semesters including a 
practicum experience. The Department adopted 
a PRT policy in December 2010. While the 
number of faculty in the department varies over 
time, the department generally has 24-26 
teaching faculty in a given year, and 6 research 
faculty. Teaching faculty are expected to teach 
at least 8 credits as well as doing research, 
while research faculty are primarily engaged in 
research and are not required to teach, though 
they may offer to teach a course in exchange for 
salary coverage. As the School of Public Health 
does not have tenure, all faculty are non- 
tenured. 

Implementation of peer review of teaching 
coincided with a school-wide redesign of the 
MPH curriculum started in 2011. The goal of this 
redesign was to provide graduates with the 
knowledge and skill sets needed for 
employment, and to assure that the program 
was adapted to the changing context in which 
public health is practiced. As many students 
were coming straight from undergraduate 
studies without much work experience, the 
redesign aimed to increase applied learning 
experiences to simulate work settings. Ideally 
students would be spending less class time 
listening to lectures and more time practicing the 
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application of public health skills with the 
instructor as mentor and coach. Five task forces 
were created at the school level to promote the 
programmatic changes: core curriculum, 
concentrations, practicum, culminating 
experience, and teaching methods and 
assessment. The teaching methods and 
assessment task force called for competency- 
based learning to promote skills which would 
improve students’ job-readiness, and highlighted 
the need to develop faculty and provide them 
with support for pedagogical changes. 

The primary objective of GH's peer review of 
teaching policy was to improve the quality of 
teaching. PRT was intended to affirm the 
importance of teaching, encourage innovation 
and improvement in teaching methods, and 
encourage sharing of teaching techniques. It 
was seen as a form of collaborative analysis that 
would help individual faculty to develop as 
teachers while reinforcing their commitment to 
teaching. Excellence in teaching would in turn 
improve the competencies of MPH graduates, 
especially in light of the move toward 
competency-based education. A second 
objective of PRT was to enable GH faculty to 
understand the courses taught by other faculty 
members so that faculty could become better 
advisors, more successfully matching students 
with course offerings that fit their needs and 
interests. Finally, the department hoped that 
PRT would help to create a stronger 
professional community in order to resist the 
centrifugal tendencies which are inevitable as 
faculty members pursue individual career goals 
and funding opportunities (d'Eon et al., 2000). 

PRT process 

The process was designed so that teams of 
two faculty members observe each course 
assigned for review. In fall and spring 
semesters, the Chair or his/her designee 
develops a list of courses to be reviewed. Every 
course is not reviewed every semester, and 
reviews are not conducted in summer to ease 
the burden on faculty. Typically a course will be 
reviewed every 2-3 times it is offered. The Chair 
or designee assigns faculty teams to review the 
courses. Factors considered in selecting 
reviewers are the background and strengths of 
reviewers and how they may be helpful to the 
instructor, and the reviewers’ ability to benefit 
from the review by gaining ideas for their own 
courses. Over time, review teams are exposed 
to many different courses. Given the inter- 


Accelerating educational innovation in the MPH degree program 


Page 45 





Journal of Health Education Teaching, 2015; 6(1): 43-56 


disciplinary nature of global health, reviewers 
often do not have content expertise in the topic 
of the course being reviewed. This is not seen 
as a problem, as the goal of PRT is to improve 
and share teaching methods, and to provide 
opportunities for collaborative reflection on 
teaching. Peer review teams are encouraged to 
meet with the instructor before observing the 
class. During this meeting they discuss specific 
aspects of the course the instructor may want to 
focus on, and review the course goals, syllabus, 
and teaching and assessment methods. This is 
meant to provide context for the observation and 
to help the reviewers see how the session to be 
observed fits into the instructor’s overall plan. 
Before conducting the observation, reviewers 
may also examine other materials related to the 
course including the course web site, e-portfolio 
site, or past student course ratings. 

Review teams schedule a time to observe the 
class. The team is expected to attend one 
(generally 3-hour) class session and to review 
the materials for the class. Given travel 
schedules for GH faculty, sometimes reviewers 
split up and review different sessions of the 
course. Following the review, the review team is 
expected to prepare a brief memorandum 
summarizing impressions and suggestions. This 
document is shared with the instructor, and the 
reviewers and instructor meet to amplify or 
clarify the comments. The memorandum is then 
submitted to the faculty chair who reviews to 
identify and prioritize resources to support 
changes in the course or skill development for 
the instructor. The review does not become part 
of the faculty member’s personnel file, nor is it 
used for annual performance review. Peer 
review tends to be more effective when the 
process is flexible and owned by the 
participating faculty (Thomas et al., 2014). To 
encourage ownership, guide questions and an 
optional rubric are provided as possible tools, 
but the review teams can decide which tools 
they want to use. The optional guide questions 
and rubric focus on aspects of course 
organization, clarity of content, interaction in the 
classroom, lecture style, discussion 
management, use of media, and exercises or 
projects to promote applied learning. 

METHODS 

We applied case study research methods to 
provide a rich description of how the PRT 
program is operated and was perceived by 
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faculty. The time period examined was January 
2011 to May 2014 (7 semesters). A combination 
of data collection methods is well suited to 
understand the relationship between 
organizational behavior and its specific context 
(Hartley, 1994). Administrative data sources 
included the school’s registrar office records 
(courses offered by the department within this 
period, and the instructors who taught the 
courses) and records from the department 
curriculum coordinator’s office (courses 
assigned for review each semester, assigned 
reviewers, number of reviews which actually 
took place, and availability of review memos). In 
addition, we used the peer review memos to 
determine the number of sessions observed for 
each course, and to help determine which 
faculty members assigned to the review actually 
participated. We also reviewed the content of all 
written peer review memos submitted during the 
study time period to elicit themes. 

To complement the administrative data and 
peer review memo information, we conducted an 
anonymous survey of GH faculty using 
Qualtrics® online software. The survey was sent 
to 26 full-time teaching faculty with regular 
appointments. The survey was sent in January 
2014, and the recall period was 2013. Faculty 
answered questions about the time spent in the 
peer review process, knowledge, and attitudes 
or opinions about the program. The knowledge, 
attitude, and opinion questions were evaluated 
on a 5 point Likert-like scale (1=strongly agree, 5 
= strongly disagree). Knowledge of the policy 
was probed with two questions: “The department 
uses peer review feedback in the faculty 
performance review and planning process,” and 
“The department uses peer review feedback in 
the curriculum revision process.” We asked 
three questions about usefulness of peer review, 
including “PRT helps me learn new methods of 
course design, teaching, or assessment,” “PRT 
increases my knowledge of course offerings,” 
and “PRT helps me to get to know other 
instructors better.” Finally, we asked opinion 
questions to probe whether the reviewer thought 
the instructor had found the feedback helpful, 
and whether the instructor found the reviewer’s 
feedback helpful. We did not ask any 
demographic information in order to protect 
confidentiality. 
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Analysis 

Administrative data were analyzed in an 
Excel spreadsheet. We calculated total and 
average per semester for: courses offered; 
courses assigned for review; reviews which 
actually took place; reviews for which a written 
memo was available. We also calculated the 
average number of reviews per course; average 
number of reviews per teaching faculty member; 
average number of sessions observed per 
review; and proportion of faculty assigned to 
review who actually participated. We 
summarized average length of memos, and the 
proportion of memos which used an evaluation 
rubric versus narrative style. 

The content of peer review memos was 
analyzed using a modified grounded theory 
approach (Corbin & Strauss, 2008). Our 
analytical strategy involved constant 
comparison, i.e. looking at observations and 
recommendations contained in each memo, and 
comparing it with observations and comments in 
other memos for similarities and differences. 
Segments of text which were found to be 
conceptually similar were grouped together 
under higher-level descriptive concepts (Corbin 
& Strauss, 2008). The lead author, who is an 
experienced researcher trained in qualitative 
methods, read each memo and entered sections 
of text into an Excel spreadsheet. These 
segments of text were then analyzed and 
assigned codes to indicate different descriptive 
themes (Miles & Huberman, 1994). 

Survey data were analyzed in Excel. Data 
were stratified by reviewers and instructors 
where necessary, and described using 
proportions and averages. 

Institutional Review Board Approval 

The study was reviewed by the university’s 
Institutional Review Board and found to be 
exempt because it does not meet the definition 
of human subject research. 

Informed Consent 

The survey used for the study was 
anonymous and responding to the survey was 
optional. We used de-identified data for this 
study and therefore we could not obtain 
informed consent of study participants. 


RESULTS 

Administrative Data 

Twenty-six faculty participated in the PRT 
program as a reviewer for at least one semester 
with 11 faculty participating as reviewers 3 or 
more times between 2011 and 2014. Each 
faculty member reviewed 2.5 courses on 
average. The GH department offered 19 courses 
and scheduled 9 peer reviews per semester 
(47%) on average. 

Over the 7 semesters, 28 unique courses 
were offered and all were scheduled for at least 
one review. Core courses with sections taught 
by different instructors were often scheduled for 
review multiple times. Of the 62 reviews 
scheduled, written memos were available for 
only 37 (60%), and verbal feedback was given 
for an additional two reviews (3%). It is assumed 
that the remaining 23 scheduled reviews (37%) 
probably did not take place due to scheduling 
problems or travel. This means that of the 28 
courses scheduled for review, 4 (14%) were 
never reviewed, 13 (46%) were reviewed once, 
9 (32%) were reviewed twice, and 2 (7%) were 
reviewed three or more times. On average each 
unique course was actually reviewed 1.3 times 
with written results. 

Some courses scheduled for team review 
were actually reviewed by only one faculty 
member. On average, 85% of faculty who were 
assigned to a reviewed course contributed to a 
review memo, and review teams observed an 
average of 1.3 sessions per course. Review 
memos were generally 1-3 pages long. Eight 
memos (22%) incorporated an optional rubric 
which rated class organization, lecture style, 
exercises, student engagement, etc. as 
“excellent”, “good”, or “needs improvement,” 
while the remaining review memos used 
narrative style. All review memos included a 
summary of observations and suggestions. 

Review Memo Content 

Reviewers provided comments on topics 
including the syllabus, choice of teaching 
methods, classroom management, lecture skills, 
and discussion management. Feedback 
depended on whether the instructor had specific 
goals which he or she wanted the reviewers to 
consider; for example, one instructor asked for 
reviewers’ recommendations for how students 
could become more engaged during class 
sessions while another instructor was interested 
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in better ways to evaluate group projects. 
Feedback on content was offered if the reviewer 
had related expertise, such as an infectious 
disease doctor reviewing a course on 
controversies in global disease control and 
eradication. Recurrent themes in the peer review 
feedback included: active learning, making 
connections between course objectives and 
activities, and teaching techniques. 

Active learning 

Reviewers gave praise and constructive 
criticism to instructors to enhance active learning 
and student engagement, as shown in these 
quotes: “The session was a great example of the 
"inside out" organization of teaching where the 
students are actively engaged in learning. Work 
outside the session (watching video lectures, 
preparing their own group's consulting 
assignment, reading another group's report) 
made it possible for this session to be so active. 
During class, groups of students evaluated 
another group's report on a drug management 
issue, and prepared and gave a 10-minute 
presentation with the findings of their 
evaluation.” “[In this session,] student groups 
had designed active-learning strategies to 
convey nutritional messages. Each group had 
just 5 minutes. One group did a charades game, 
another used a polling software to survey class 
participants through text messaging, a third 
group had written a song with nutrition 
messages in the lyrics...the students seemed 
happy and engaged.” Active learning sometimes 
involved individual activities such as students 
filling out a worksheet or team activities 
(presentations, group projects) inside or outside 
the classroom. One reviewer acknowledged a 
faculty member’s innovation in incorporating 
tweeting into the classroom while also 
questioning how to evaluate the impact of 
tweeting on educational outcomes. Other 
reviewers suggested strategies such as pair- 
shares, “clickers” (audience response system), 
“minute papers” (a classroom assessment 
technique to get rapid feedback on a main idea 
introduced in the session), and structured 
discussions to further engage students. 

Connections and context 

Another theme of reviewer comments was 
helping learners see connections and 
understand the context in which course 
concepts can be applied in professional practice. 
Reviewers praised instructors for using 
techniques which put the session in the context 
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of the overall course goals. They also suggested 
additional readings or perspectives connecting 
other disciplines: for example, a professor with a 
humanities degree, after reviewing a course 
session on vaccine refusal taught by a medical 
doctor, ruminated that John Stuart Mill would 
ultimately come down on the side of compulsory 
vaccination without exclusions because not 
vaccinating compromises the health and liberty 
of others. The reviewer supplied a lengthy quote 
from Mill’s book On Liberty to support this 
argument. 

Frequently, reviewers observed ways in 
which the instructor could make explicit the 
connections between sessions, summarize key 
points, or anticipate what is coming next as in 
these quotes: “It might be good to introduce the 
readings for the day by saying how or why they 
were chosen, or what the students are meant to 
get out of them. Are they linked in any way; an 
overarching theme?” “I don't usually subscribe 
to the philosophy that one needs to hit students 
over the head with the course objectives, but it 
might have helped [if you would] place the 
discussion in the overall context of the course 
and give objectives for the lecture. ” 

Teaching styles and techniques 

Reviewers observed ways in which the 
instructor set up the classroom to enable active 
discussions, intervened to prevent domination 
by vocal students, and used techniques such as 
randomly picking students to lead discussion as 
a way to motivate students to be more prepared 
and therefore more able to actively participate. 
Reviewers mentioned ways to widen 
participation, suggesting techniques such as 
coaching student discussion leaders and 
preparing discussion questions in advance. 

Reviewers gave feedback on systematic 
organization of lectures, confidence and 
command of material, the instructor’s voice and 
enthusiasm, and the preparation of slides. 
Reviewers liked when instructors integrated 
examples and experience into lectures, as 
shown in these quotes: “One of the most 
effective elements was when the instructor 
turned the lecture from conceptual to the specific 
by integrating data from recent scientific 
publications which were thematically linked to 
the student projects. ” “The instructor spoke from 
experience, probably the greatest strength of the 
class. It really addresses, ‘what is it like to work 
in the field?’” Many reviewers made suggestions 
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about how to design better presentations, use 
questions more effectively, and wait an 
adequate amount of time for students to answer. 

Other themes 

Reviewers often commented on themes ripe 
for deeper discussion among faculty, such as 
whether instructors should ‘cold call’ on quiet 
students, the appropriate role of teaching 
assistants in lecturing, whether to allow re¬ 
submissions of work for a higher grade, how to 
assure individual accountability in group 
projects, and what to do about students who are 
web surfing or emailing in class. While reviewers 
acknowledged the complexity of these issues 
and there is likely no one right way, they would 
often mention their own preferences as these 
reviewers did on the topic of grading: “I wonder if 
students will be happy doing so much [group] 
work and getting few individual grades, but that 
may just be my prejudice against group projects. 
I didn't like them as a student and did not find 
them useful when I tried them as a teacher. ” “If 
we are really interested in the intellectual growth 
of our students ...then we would want to 
encourage them to redo work, continue to 
practice, and if that produces a better product, 
reward them accordingly. ” 

Survey Data 

We received responses from 19 of 26 GH 
faculty surveyed (73% response rate). Fifteen 
(79%) of the respondents had participated in 
PRT in 2013. Where reviews were not 
completed, faculty explained reasons including 
scheduling problems (being asked to review a 
course that met when the reviewer was 
teaching), international travel, and “running out 
of time” or forgetting. Reviewers reported 
spending a median of 4 hours while instructors 
said they spent 0.5 hours on the peer review 
(instructors did not count the time being 
observed, as they would have been teaching 
anyway). 

Most of the faculty surveyed said that when 
they had reviewed a course, they had met with 
the instructor, reviewed the syllabus, observed a 
session, and shared written feedback with the 
instructor afterwards (Figure 1). Less common 
peer review activities were talking to students to 
get their feedback and reviewing the course web 
site or past course evaluations which are 
available to all faculty and students online. In 
one case, a pair of reviewers had conducted an 
online survey to get student feedback. 
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About three-quarters of instructors believed 
that peer review was important for faculty 
development and 44% of instructors reported 
they had already made changes in course 
design or teaching based on peer review 
feedback. Faculty shared their positive 
perceptions, as in these quotes: “I think this 
process helps knit the department together.” “I 
highly value peer review. I’ve made changes 
based on comments from previous reviews.” “I 
enjoyed having my colleagues in the classroom. 
I always ask them to participate and that adds a 
nice new voice to the session.” Other faculty 
voiced doubts about the usefulness: “[The 
usefulness] depends on the feedback provided. 
Teaching styles vary and not all 
recommendations are helpful.” “The reviewer 
came—but so far has not responded to request 
for feedback.” Fifty-three percent of instructors 
thought peer review of their course was 
worthwhile or very worthwhile. 

Attitudes toward peer review suggest that the 
reviewers gain as much if not more from the 
peer review process as the instructor whose 
course is being reviewed (Table 1). Almost 
three-quarters of reviewers said that conducting 
peer review helped them to learn new design, 
teaching and assessment methods. Reviewers 
also said they gained knowledge of the range of 
course offerings in their department and got to 
know other faculty better through the peer 
review process, as these quotes suggest: “I 
enjoy doing the reviews. I especially like meeting 
with the instructors ahead of time and finding out 
what they like or are worried about in their 
courses. I like to see how other instructors 
structure their courses. ” “I value the opportunity 
to learn from my colleagues and offer feedback. 
While it can be hard to fit in our busy schedules, 
it’s always worth it. ” 

Some respondents didn’t know how their 
feedback had been perceived or suspected that 
the feedback they gave to instructors was not 
welcome. In addition, reviewers were not aware 
of how the department was using the peer 
review feedback: only 27% believed the 
department uses the peer review memos in the 
curriculum revision process (a stated goal of the 
peer review policy) while the same percent 
believed that the department used the peer 
review feedback in the faculty performance 
review and planning process (the policy explicitly 
states that the peer review process is not 
evaluative). 
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CONCLUSIONS 

Through this case study, we documented and 
analyzed the implementation of a PRT program 
meant to further MPH curriculum re-design goals 
to emphasize competency-based learning. We 
found that the peer review process was 
generally, though not uniformly, well received by 
faculty. The proportion of instructors who said 
they had already made changes in course 
design or teaching as a result of PRT (44%) was 
similar to that found in other studies; for 
example, about one-third of participants in a 
PRT program at the University of Nebraska 
reported making significant changes to some 
components of teaching as a result of the 
program (Bernstein et al., 2000). The content of 
review memos was consistent with the stated 
goals of the policy, i.e. to encourage innovation 
and improvement in active teaching and learning 
methods, help faculty develop as teachers and 
advisors, and to create a strong community. 

One theme emphasized by reviewers was 
the need to include more practice-based 
examples in lectures. This suggests that public 
health faculty recognize that practice experience 
is highly valued by graduate students, most of 
whom are being trained for practice rather than 
research careers or academia. Integrating 
examples and stories from applied practice 
helps students to see the relevance and 
possible applications of theoretical concepts. 
While not, in itself, active learning, this may help 
make the material more stimulating and 
interesting which is a quality of good teaching 
(Thomas et al., 2014). Reviewers’ specific 
criticism of lecture styles also seemed intended 
to help instructors increase engagement with 
students at their level of understanding 
(Wilkerson & Irby, 1998). 

A large number of comments from reviewers 
related to how instructors could strengthen the 
connection between the content of the course 
and the situations in which students might be 
expected to use or apply the content as public 
health professionals. In active-learning courses, 
this is a challenge because the key points are 
emerging in real-time from the class. Faculty 
members who are innovating with active 
learning may not yet be fully competent at aiding 
students’ sense-making when the activity 
concludes (Ketelaar, Beijaard, den Brok, & 
Boshuizen, 2013). PRT provides an opportunity 
to coach faculty in this difficult skill. 
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As has been noted in other studies, the PRT 
process provided opportunities for faculty 
development not only for the instructor but for 
the reviewer as well (Kohut et al., 2010; 
Wilkerson & Irby, 1998). In one study, reviewers 
perceived the process more useful for their own 
teaching than did faculty being reviewed; 
however, the PRT process in that study was 
being used for evaluative purposes to make 
tenure and promotion decisions, a different 
context (Kohut et al., 2010). Observing someone 
else is a way to self-critique and learn new 
practice (Chism, 2007), and peer review helps 
faculty interact and learn from one another, 
providing collegial support (d'Eon et al., 2000). 
The GH PRT program shows that this was an 
important outcome: 78% of reviewers and 72% 
of instructors noted that the peer review process 
helped them get to know other faculty as 
colleagues. The increased knowledge of other 
teachers’ courses and the social ties created 
through PRT are resources which can be used 
in developing MPH curriculum integration and 
making connections between coursework and 
public health practice. Over 80% of reviewers 
noted that PRT helped them understand the 
Department’s courses, which helps them 
become better advisors to students. In addition, 
the GH department has added “teaching 
innovation” as a standing agenda item at 
monthly faculty meetings. This additional forum 
for communicating about teaching aligns with 
the PRT goals and further reinforces the 
teaching community. It has helped faculty 
identify in-house experts for particular 
innovations, and has helped to emphasize the 
value placed on educational innovation. 

In addition to providing feedback to individual 
teachers, the PRT process highlighted more 
general issues or questions which leaders may 
want to address in the context of school-wide 
MPH redesign. Topics included the value of 
competency-based grading (i.e. allowing 
students to re-submit work until they achieve 
competency, and only grading the final product), 
how to engage introverted or shy students and 
students from diverse cultural backgrounds, and 
how to minimize technology distraction while 
also meeting students where they are, i.e. 
incorporating social media and other new 
technology into the classroom. It would be 
helpful to provide more opportunities for the 
faculty as a whole to discuss such observations 
and issues. PRT helps to promote active, 
practice-based learning because it implicitly 
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recognizes that teaching is not merely a 
technical enterprise, but a social practice (d'Eon 
et al., 2000). By allowing the reviewer and the 
instructor to define the process, valuing the 
process of reviewing as well as receiving 
feedback, and providing multiple opportunities to 
engage in reflection, PRT helps facilitate the 
adoption of new social practices around 
learning, such as project-based learning. 

Limitations 

This study has several limitations. Although 
we had a high response rate to the online 
survey, not all faculty responded and we cannot 
know how the perceptions and experience of 
non-respondents might have differed. The 
thematic analysis drew on written peer review 
memos, which were only provided for 60% of 
scheduled reviews. Finally, our analysis of a 
single case study may not reflect the experience 
of other schools or departments. However, the 
framework that resulted may help guide the 
design of PRT programs in other settings, as 
discussed below. 

RECOMMENDATIONS 

Based on our review of PRT literature and 
this case study, we have created a framework 
which captures our experience and may guide 
other schools of public health considering 
adopting PRT as part of the redesign of MPH 
programs (Table 2). The framework illustrates 
pre-disposing factors, policy design, and 
program implementation features, and how 
together they may lead to the intended 
outcomes of PRT. 

Pre-disposing Factors 

Positive perceptions of PRT are correlated 
with collegiality and good relations among 
reviewers and observers (Byrne, Brown, & 
Challen, 2010). Power dynamics and fear can 
have an important influence on PRT 
implementation and outcomes (Byrne et al., 
2010; Kell & Annetts, 2012; Kohut et al., 2010). 
Our study was conducted in a non-tenure 
situation, which may have reduced power 
imbalances as we did not need to consider the 
social relationships among tenured and non- 
tenured faculty when assigning reviews or 
sharing review feedback. The absence of tenure 
may have made it easier for faculty to see the 
review process as developmental rather than as 
an evaluation tool (Peel, 2005). In a tenure 
system, being very clear about the formative 
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purpose of PRT and taking action to reduce 
power imbalances in pairing of reviewers and 
instructors is even more important to ensure 
acceptance of the process. 

Another factor associated with successful 
PRT is whether faculty have established norms 
of shared inquiry and prior experience working in 
teams (Thomas et al., 2014). Research has 
shown that in disciplines where faculty naturally 
engage in individual research with sole 
authorship, PRT may be less successful 
(Thomas et al., 2014). Our case study was 
conducted in a department which was already 
used to working in interdisciplinary teams for 
research projects, which may have made it 
easier to apply a team approach to reflect on 
teaching. In situations where team work among 
faculty is not the norm, leaders may need to 
spend more time in forming teams and 
developing teamwork and reflective practice 
skills to ensure PRT works well. 

Policy Design 

Leadership is essential to motivate faculty 
especially in public health schools with soft 
funding where faculty are juggling the need to 
cover their salary by generating research dollars 
and may see teaching as less essential for 
survival. Time and burden on faculty are 
consistently raised as issues affecting PRT 
implementation success (Kell & Annetts, 2012; 
Siddiqui, Jonas-Dwyer, & Carr, 2007; Thomas et 
al., 2014). While the model of peer interaction 
and consultation adopted in this case study was 
not very intensive-requiring about a half-day of 
each participating faculty member’s time per 
semester and minimal administrative support- 
the program experienced compliance problems: 
40% of review teams either did not submit 
written memos or did not conduct the assigned 
review, and several comments from faculty 
mentioned the time burden. Leadership is 
needed to signal that committing time to the 
process is valued and rewarded. 

Leadership is also needed to monitor and 
evaluate how PRT initiatives contribute to the 
outcomes of student learning and professional 
development. Closer management with more 
frequent reminders and assistance in scheduling 
could also help increase post-observation 
written feedback from reviewers and facilitate 
deeper, more meaningful discussion of 
observations. Leadership must also make sure 
that the structure for PRT is focused and 
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periodically reinvigorated to avoid stagnation 
and complacency (Byrne et al., 2010; Kell & 
Annetts, 2012; Kohut et al., 2010). For example, 
to re-invigorate a moribund PRT program, one 
university introduced year-long peer 
development team projects, where 2-5 teaching 
staff worked together on a project to improve 
practice such as effective use of tutorial support, 
planning and developing a new course, or 
adapting pedagogy to use interactive 
whiteboards (Byrne et al., 2010). Kell and 
Annetts (2012) found changing teams was 
helpful so conversations didn’t stagnate and 
reflection continued to be constructive. 

The developmental focus of the PRT 
program must be clearly communicated in order 
to reduce fear and foster development of 
reflective practice (d'Eon et al., 2000). Even 
though PRT in this case study was formative 
and not evaluative, there were misconceptions 
among faculty. For example, about a third of 
faculty believed the department uses the peer 
review feedback in the faculty performance 
review process, which is not the case. Ensuring 
faculty are clear on the purpose of the PRT may 
increase engagement with the process. 

In addition, faculty ownership of the process, 
through opportunities to choose tools and 
approaches and to influence the way reviews 
are carried out, increases positive perceptions of 
PRT and may enable a more critical and 
collegial examination of teaching practice 
(Thomas et al., 2014; Toth & McKey, 2010). We 
fostered faculty ownership through opportunities 
to choose rubrics, reporting style, specific 
session or sessions to attend, and the types of 
other information to consider (e.g. student 
evaluations, web sites). 

A factor which has impeded PRT in some 
studies is uncertainty about what should be 
reviewed (Thomas et al., 2014). Some PRT 
processes use external criteria, while others 
base the assessment on goals set by the 
reviewer and instructor (Siddiqui et al., 2007). 
The GFI department’s PRT policy did not define 
“good teaching” criteria in advance other than 
suggesting that improved teaching should 
promote competencies that MPH graduates 
need in the work place. Review teams were 
allowed to decide which aspects of teaching to 
assess, and how to interpret and apply the goal 
of competency-based learning. As in other 
studies (Courneya et al., 2008), we observed 
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that reviewers were influenced by their own 
perspectives on teaching, e.g. an aversion to 
group projects. In addition, our finding that some 
reviewers were uncertain whether their advice 
was useful suggests that it might have been 
helpful to discuss criteria for good teaching to 
develop a common understanding. This need 
not reduce the flexibility afforded the faculty in 
the tools and process used for review. In fact, 
initial cycles of PRT could be used to construct 
evidence of qualities of effective teaching (Drew 
& Klopper, 2014). 

Program Implementation 

A key question in PRT program 
implementation is who is qualified to be a peer 
reviewer (Thomas et al., 2014). One study 
determined that feedback received from subject 
specialists and non-specialist assessors was 
equally reliable and valid (Hanson, 1993). Our 
investigation provides further evidence that 
reviewers with different disciplinary backgrounds 
can still provide useful formative feedback and 
ideas to stimulate critical reflection. While PRT 
can help assess relevancy of course content, 
the purpose of formative PRT is much more 
about encouraging reflective practice-related 
dialogue, sharing ideas about specific pedagogic 
innovations, and sharing constructive feedback 
that can lead to new understanding and 
improved practice (Kell & Annetts, 2012; 
Thomas et al., 2014). In order to achieve these 
goals, the program needs to assign reviewers to 
maximize exposure of individual faculty to 
different styles and approaches to teaching. 

Training of peer reviewers may also be 
beneficial. Although the GH PRT did not provide 
specific training to peer reviewers, some studies 
have shown that trained reviewers may be more 
accurate observers and insightful of their own 
capabilities, thus establishing trust (Carolan & 
Wang, 2012; Drew & Klopper, 2014; Kohut et 
al., 2010; Thomas et al., 2014). Training should 
recognize that peer review is not just a way to 
identify mechanical “tips” to improve teaching, 
but rather it is a service activity to foster 
questioning of beliefs, assumptions, and habitual 
practices around teaching (d'Eon et al., 2000; 
Thomas et al., 2014). 

Program implementation requires special 
attention to create administrative tools to keep 
track of reviewer assignments and reports, in 
order to assure equal coverage of courses, and 
diversity and breadth of experience for 
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reviewers. This becomes especially important if 
peer review is to be implemented across units to 
increase opportunities to work collaboratively 
(Thomas et al., 2014). 

Finally, leaders should make explicit efforts to 
connect PRT to other opportunities to engage 
and enrich teaching community, creating a more 
comprehensive approach to faculty 
development. Once the GH department had 
added teaching innovation as a standing item on 
the agenda of faculty meetings, topics discussed 
included use of online software for in-class polls, 
quizzes, and to take attendance 
(http://socrative.com/) ; software for cropping 
segments of longer video and inserting quiz 
questions or commentary (https://edpuzzle.com/) . 
and an innovate course design which teaches 
students about approval of new medicinal 
products through a game strategy. At the school 
level, workshops are organized to discuss and 
promote innovative teaching methods. Other 
ways to strengthen connections include aligning 
faculty development with policies and processes 
for rewarding excellence in teaching, such as 
awards or grants for teaching projects (Drew & 
Klopper, 2014). Schools could encourage 
technique-specific coaching relationships among 
faculty or provide incentives for faculty to 
collaborate on new course development as they 
collaborate on research grants (Frenk et al., 
2015). A web site might be created to share 
teaching tools and methods, and opportunities to 
discuss cross-cutting teaching issues could be 
built into meeting schedules. These activities 
expand the idea of PRT and connect it to the 
larger goal of promoting a supportive faculty 
learning community and an organization that 
values education (Cox, 2004). 

Outcomes 

Our framework suggests that PRT can lead 
to outcomes including increased opportunities to 
reflect on the practice of teaching and increased 
motivation to learn about teaching (Peel, 2005; 
Thomas et al., 2014), thus giving faculty greater 
confidence in their ability to adapt teaching 
methods to the changing landscape of MPFI 
education. The results of this case study and 
prior studies show PRT can be beneficial to 
reviewers as well as instructors (Costello et al., 
2001) by helping them learn new design, 
teaching and assessment methods and become 
better advisors to students. Schools should 
consider adapting the PRT process to 
encourage reflection on criteria of good 
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teaching, and to explicitly recognize the benefits 
to both reviewers and instructors. PRT has the 
potential to change the social practice of 
teaching within MPFI education, to re-orient 
normative standards toward more active, 
practice-based learning. This is the important 
role that peer review can play in accelerating 
educational innovation in training MPH students 
for careers as public health professionals. 
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Table 1: Instructors’ and reviewers’ attitudes about the peer review process 

Strongly agree or agree 


Instructors _ Number Percent 

Gained new insights about course design, teaching methods, or 8 47 

assessment (n=17) 

Having course reviewed was worthwhile (n=17) 9 53 

Peer review is important for faculty development (n=18)13_72 


Reviewers _ 

Learned new methods of course design, teaching or assessment (n=18) 13 72 

Have greater knowledge about Department’s course offerings (n=18) 15 83 

Helped with getting to know other instructors in the department (n=18)14_78 


Table 2: MPH Peer Review of Teaching Framework: Policy Design/Program Implementation 
Features and Desired Outcomes 


Pre-disposing Policy design Program implementation Outcomes 

factors 

• Prior experience 
conducting team- 
based research 

• Collegial 
environment 

• Leadership 
support 

• Faculty ownership 

• Developmental 
focus 

• Common 
understanding of 
the characteristics 
of good teaching 

• Flexible 
tools/processes to 
encourage shared 
critical reflection 

• Does not require specialist 
reviewers 

• Assigns reviewers in ways 
that broadens perspectives 
and experiences over time 

• Provides training on how to 
review and make meaning 
from review process 

• Creates tools to track 
assigned and reviewed 
courses, and monitor that 
process is being followed 
and results are being 
achieved 

• Links PRT process to other 
opportunities to engage 
and enrich teaching 
community 

• Increased knowledge of 
whole curriculum 

• Increased reflective 
practice 

• Increased motivation to 
learn about teaching and 
confidence in ability to 
adapt teaching to context 

• Strengthened teaching 
community 

• Re-oriented normative 
standards of teaching 
towards active, practice- 
based learning 
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Figure 1: Peer review activities mentioned by reviewers (N=16) and instructors (N=13) 
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